Skip to content

Conversation

@frankmcsherry
Copy link
Member

@frankmcsherry frankmcsherry commented Mar 24, 2025

Slowly building up more confidence in changing our internal operator reachability implementation. This is a more careful version of #651, which in particular avoids a shadowing bug that seemed to be at the root of #654.

The external facing changes are that customer operator input/output creation no longer take a Vec<Antichain<_>>, but rather an I: IntoIterator<Item=(usize, Antichain<_>)>. This allows a sparse description of connectivity, in cases where it would be expensive to produce a large number of Antichain::default() empty antichains.

The new behavior is scale-tested in tests/shape_scaling.rs, where we test operators and subgraphs up to 100,000 x 100,000.

@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch from 7d8c5c4 to 4929372 Compare March 24, 2025 21:44
@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch 2 times, most recently from fcbac30 to b4ef9b2 Compare March 25, 2025 15:36
@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch from b4ef9b2 to f8a234a Compare March 25, 2025 15:49
@frankmcsherry frankmcsherry marked this pull request as ready for review March 26, 2025 02:27
Copy link
Member

@antiguru antiguru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good; left one inline comment.

@frankmcsherry frankmcsherry merged commit b43381b into TimelyDataflow:master Mar 26, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants